Skip to content

fix: add SSE backpressure safeguards for slow clients#19493

Closed
sjawhar wants to merge 1 commit intoanomalyco:devfrom
sjawhar:fix/sse-backpressure
Closed

fix: add SSE backpressure safeguards for slow clients#19493
sjawhar wants to merge 1 commit intoanomalyco:devfrom
sjawhar:fix/sse-backpressure

Conversation

@sjawhar
Copy link
Copy Markdown

@sjawhar sjawhar commented Mar 28, 2026 •
edited
Loading

Issue for this PR

Fixes #16697

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Adds backpressure to the AsyncQueue used by SSE endpoints to prevent unbounded memory growth when clients fall behind.

Root cause: AsyncQueue has no size limit. When an SSE client stalls (slow network, backgrounded tab, stalled proxy), Bus.subscribeAll keeps pushing JSON.stringify'd events into the queue without bound. In production this caused 187GB RSS.

Fix: Add optional capacity parameter to AsyncQueue with drop-oldest behavior. When the queue exceeds capacity, oldest items are discarded. Set to 1024 items (~2MB at ~2KB avg event size) for both SSE endpoints. TUI queues remain unbounded since they are 1:1 request/response pairs that never accumulate.

The web UI already handles reconnection gracefully (full state reload on server.connected), so dropped events during a stall are recovered naturally.

How did you verify your code works?

  • Bus and snapshot tests pass
  • Verified queue drops oldest items when capacity exceeded
  • Existing SSE consumers (TUI) unaffected — no capacity parameter means unbounded

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

@github-actions github-actions bot added needs:compliance This means the issue will auto-close after 2 hours. needs:issue labels Mar 28, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 28, 2026

Thanks for your contribution!

This PR doesn't have a linked issue. All PRs must reference an existing issue.

Please:

  1. Open an issue describing the bug/feature (if one doesn't exist)
  2. Add Fixes #<number> or Closes #<number> to this PR description

See CONTRIBUTING.md for details.

@sjawhar sjawhar mentioned this pull request Mar 28, 2026
Closed
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 28, 2026

The following comment was made by an LLM, it may be inaccurate:

Potential Duplicate PRs Found

1. #19423 - fix(server): add backpressure to SSE event queues

2. #18538 - fix(opencode): handle client disconnect in SSE event route writes

@github-actions github-actions bot removed needs:compliance This means the issue will auto-close after 2 hours. needs:issue labels Mar 28, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 28, 2026

Thanks for updating your PR! It now meets our contributing guidelines. 👍

@sjawhar
Copy link
Copy Markdown
Author

sjawhar commented Mar 28, 2026

Note on PR#19423

PR#19423 appears to be a duplicate of this PR — both implement the same SSE backpressure safeguard fix for issue #16697.

Relationship:

  • PR#19423: Already merged/approved with proper template format and Closes #16697
  • PR#19493: This PR, rebased version with same fix

Recommendation: If PR#19423 is already merged, this PR can be closed as duplicate. If not, one should be closed in favor of the other to avoid duplicate commits.

@github-actions github-actions bot mentioned this pull request Mar 29, 2026
Closed
6 tasks
@sjawhar
Copy link
Copy Markdown
Author

sjawhar commented Apr 2, 2026

Closing: consolidated into the serve hardening branch on sami.

@sjawhar sjawhar closed this Apr 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Multiple memory leaks cause unbounded RAM growth during extended TUI usage

1 participant

@sjawhar